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Abstract 



The Johnson-Lindenstrauss lemma is a fundamental result in probability with several ap- 
plications in the design and analysis of algorithms in high dimensional geometry. Most known 
constructions of linear embeddings that satisfy the Johnson-Lindenstrauss property involve ran- 
domness. We address the question of explicitly constructing such embedding families and provide 
a construction with an almost optimal use of randomness: For < (5, er < 1, we give an explicit 
generator G : {0, 1}'' ^ for s = 0{log{l/S)/e^) such that for all w € K", ||w|| = 1, 

Pr [\\\G{y)w\\^-l\>e]<6, 

and seed-length r = O (log{n/5) ■ log |^l2Slli/^i^ ^ in particular, for 5 = l/poly(?i) and fixed 

e > we get seed-length 0(log?iloglog7i). Previous constructions required at least O(log^n) 
random bits to get polynomially small error. 



1 Introduction 



The celebrated Johnson-Lindenstrauss lemma (JLL) [JL84] is by now a standard technique for 
handling high dimensional data. Among its many known variants (see [AV99, DG03, IM98, Mat08]), 
we use the following version originally due to Achlioptas [Ach03] as reference ^. 

Theorem 1.1 (Achlioptas). For all w G M", = 1, e > 0, s = Clog(l/5)/e^, 

Pr [\\\il/^)Awf -l\>e]<5. 

We say a family of random matrices has the JL property (or is a JL family) if a similar condition 
holds. In typical applications of JLL, d is taken to be l/poly(n) and the goal is to embed a given 
set of poly(n) points in n dimensions to O(logn) dimensions with distortion at most e for a fixed 
constant e. This is the setting we concern ourselves with. 

Most results on embedding the Euclidean space as above are probabilistic in nature. However, 
a simple probabilistic argument shows that there exists a fixed collection of poly (n, 1/5) linear 
mappings satisfying the JL property. Despite much attention, the best known constructions of JL 
families use at least 0(lognlog(l/(5)) random bits [CW09]. Besides being a natural problem in 
geometry as well as derandomization, an explicit family of Johnson-Lindenstrauss transformations 
would likely help derandomize other geometric algorithms and metric embedding constructions. 
Further, having an explicit construction is of fundamental importance for streaming algorithms as 
storing the entire matrix (as opposed to the randomness required to generate the matrix) is often 
too expensive in the streaming context. 

Our main result is an explicit generator that takes roughly 0(log7iloglogn) random bits and 
outputs a matrix A G M*^" satisfying the JL property. 

Theorem 1.2. For every < e,5 < 1, there exists an explicit generator G : {0, 1}'' — t- M"^^ for 
s = Clog(l/5)/e^, such that for every w G M", = 1, 

Pr [\\\G{y)w\\^ -l\> e]<5. 
s/Gu{o,i}'- 

The seed-length of the generator is r = O (log{n/5) • log ^ ^"^^"/"^^ 

Our construction is elementary in nature using only standard tools in derandomization such as 
/c-wise independence and oblivious samplers [Gol97]. The construction has the additional property 
that for 6 = l/poly(n), the matrix- vector product G{y)w can be computed efficiently in time 
0(?7-logn). The computational efficiency does not follow directly from the dimensions of G{y), as 
in our construction G{y) is obtained by composing several matrices some of which are of dimension 
0{^/n) X n. Nevertheless, the large matrices are obtained from the discrete Fourier transform 
matrix facilitating fast matrix- vector product computations. 

Further, as one of the motivations for derandomizing JLL is its potential applications in stream- 
ing, it is important that the entries of the generated matrices be computable in small space. We 
observe that for any i G [s], j G [n], y G {0, 1}^', the entry G{y)ij can be computed in space O(logn) 
and time 0{n'^~^°^^^) (for fixed e, 6 > l/poly(ri)). (See proof of Theorem 1.2 for the exact bound) 



^Throughout, C denotes a universal constant. For a multiset S, x £u S denotes a uniformly random element of S. 
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1.1 Related Work 



Independently, Kane and Nelson [KNIO] obtained a construction that is similar in spirit to ours 
and achieves a slightly better seed-length of r = 0(logri + log(l/(5) log(log(l/(^)/e)). Note that for 
the important case of 6 polynomially small, our seed-length is the same as theirs. 

The £2 streaming sketch of Alon et al. [AMS99] implies an explicit JL family with seed-length 
O(logn) for embedding M" into with distortion e and error 6, where s = 0{l/e'^5). Karnin et 
al. [KRS09] construct an explicit family for embedding into with distortion e = l/s"*^ and 
error S = 1/ s~^. The seed-length they achieve is (1 -|- o(l)) log n + 0(log^ s). 

The works of Diakonikolas et al. [DKNIO] and Meka and Zuckerman [MZIO] construct pseudo- 
random generators for degree 2 threshold functions achieving a seed-length of logn • poly(l/(5) for 
fooling with error at most 5. As derandomizing the JL lemma is a special case of fooling degree 2 
PTFs, these works give a JL family with seed-length logn • poly(l/5). 

The best known explicit JL family is the construction of Clarkson and Woodruff [CW09] who 
show that a random scaled Bernoulli matrix with 0(log(l/5))-wise independent entries satisfies the 
JL lemma. We make use of their result in our construction. 

We also note that there are efficient non-black box derandomizations of JLL, [EIO02], [Siv02]. 
These works, take as input N points in W^, and deterministically compute an embedding (that 
depends on the input set) into M'^(^°sJV)/e -v^rhich preserves all pairwise distances between the given 
set of N points. 

Finally, we remark that our goal as well as result is very different from those of the recent works 
[AC09, AL09, DKSIO, ALIO, KNIO] on fast or sparse Johnson-Lindenstrauss transformations as 
pioneered by the seminal work of Ailon and Chazelle [AC09] . The goal in these works is to design a 
family of embedding matrices for which the matrix-vector products Ax can be computed efficiently 
(usually O(nlogn)) and are mainly concerned with the setting where the desired error probability 
6 is exponentially small. In contrast, we are mainly interested in the case where 5 is polynomially 
small but want to save on randomness. 

1.2 Outline of Construction 

Our construction is based on a simple iterative scheme: We reduce the dimension from n to 0{^/n) 
using /c-wise independence and oblivious samplers [Gol97] and iterate for O(loglogn) steps. 

Fix a vector w S M"' with \\w\\ = 1 and let 6 = l/poly(n). We first use an idea of Ailon and 
Chazelle [AC09] who give a family of unitary transformations TZ from M" — )• R" such that for every 
w G M" and 1/ G„ 7^, the vector Vw is regular in the sense that ||Fw||oo = 0(\/log n/n). We 
derandomize their construction using limited independence to get a bound of HT^it'lloo = 0{n~^^^). 

We next observe that for a vector w G M", with ||i(^||oo = 0('^~"'^^^||tw||2) projecting onto a 
random set of 0(n^/^ log(l/(5)/e^) coordinates preserves the £2 norm with distortion at most e. We 
then note that the random set of coordinates can be chosen using efficient samplers as in [Gol97]. 
The idea of using samplers is due to Karnin et al. [KRS09] who use samplers for a similar purpose. 

Finally, iterating the above scheme O (log logn) times we obtain an embedding of M" to ]RP°^y^°g" 
using roughly O (log n log log n) random bits. We then apply the result of Clarkson and Woodruff 
and perform the final embedding into 0{log{l/6) /e^) dimensions by using a random scaled Bernoulli 
matrix with 0(log(l/(5))-wise independent entries. 
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2 Preliminaries 



Let Hn G {—l/\/n, 1/ y/n}^^'^ be the normalized Hadamard matrix such that H^Hn = In (we drop 
the suffix n when dimension is clear from context). While the Hadamard matrix is known to exist 
for powers of 2, for clarity, we ignore this minor technicality and assume that it exists for all n. 
We make use of Khintchine-Kahane inequalities (cf. [LT91]). 

Lemma 2.1 (Khintchine-Kahane). For every w G M", x {1, —!}"■, k > 0, 

E[\{w,x)\''] < k''/^E[\{w,x)\'^f/^ = k''/^\\wf. 
We use efficient oblivious samplers as in [Gol97]. 

Theorem 2.2. For every e,6 there exists s = Clog(l/(5)/e^ such that the following holds. There 
exists an explicit collection of subsets of [n], S{n,e,6), with each S £ S of cardinality \S\ = s, and 
\S\ = n ■ poly(l/e, 1/S) such that for every function / : [n] — ?• [0, 1], 



Pr 

SGuS 



-j;/(i)-. E fii) 



ies 



> e 



< 6. 



Corollary 2.3. For every e,6,B > there exists s = Clog(l/5)i?^/e^ such that the following holds. 
There exists an explicit collection of subsets of [n], S{n,B,e,5), with each S £ S of cardinality 
\S\ = s, and \S\ = n ■ poly(i3/e, 1/5) such that for every function / : [n] — >■ [0, B], 



Pr 



G ^ ^ 



> e 



< 6. 



Proof. Follows by taking S = S{n, e/B, 6) as in Theorem 2.2 and using the condition of Theorem 2.2 
for f:[n]^ [0, 1] defined by f{i) = f{i)/B. □ 

Finally, we use the following result of Clarkson and Woodruff. 

Theorem 2.4 (Theorem 2.2, [CW09]). There exist constants c, C such that the following holds. For 
< e, 5 < 1, s = clog(l/5)/e^, let A G W^^"^ be a random matrix with entries in {—1/^/s, l/^/s} 
that are {C log{l/ 5)) -wise independent. Then, for every w G M", \\u}\\ = 1, 



Pr[ 

A 



l\> e]<6. 



Note that for all k, m, constructions of /c-wise independent spaces over {1, —1}™ with seed-length 
0(A;logm) are known. 



3 Main Construction 

Suppose that 6 > for a constant c > 0. If not we first embed the input vector into M.'^ for 

N = [1/5] by retaining the first n coordinates and setting the other coordinates to be 0. The 
parameters we get by working over will be the same as those of Theorem 3.4 and hence of 
Theorem 1.2. Further, we assume that log(l/(5)/e^ = o(n) as else JLL is not interesting. 

As outlined in the introduction, we first give a family of rotations to regularize vectors in M". 
For a vector x G M", let D{x) G M"^" be the diagonal matrix with D{x)ii = Xj. 
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Lemma 3.1. Let x G {1,-1}" he drawn from a k-wise independent distribution. Then, for every 
w eM."" with \\w\\ = 1, 



Pr[\\HD{x)w\ 



> n 



-(l/2-a) ■ 



< 



n 



ak~l ' 



Proof. Let v = HD{x)w. Then, for i £ [n], Vi = HijXjWj and E[v~] = ^jH^j^j 
Markov's inequality and Khintchine-Kahane inequality Lemma 2.1, 

The claim now follows from a union bound over i £ \n\. 



1/n. By 



□ 



We now give a family of transformations for reducing n dimensions to 0(n^/^) dimensions with 
distortion at most e. For S Q [n], let Vs '■ IR" — ?• M'"^' be the projection onto the coordinates in S. 

Lemma 3.2. Let S = S{n,n^/^,e,6), s = 0(n^/^ log(l/(5)/e^) be as in Corollary 2.3 and let 
D be a k-wise independent distribution over {1,-1}". For S €„ S , x ^ D , let random linear 
transformation As^x ■ 1^" — ^ be defined by As^x = \l n/s ■ Vs ■ HD{x). Then, for every w S 
with \\w\\ = 1, 

Pr[\\\As^x{w)f - l\ >e] < 5 + k^'"^ /n^l^^^ . 
Proof. Let v = HD{x)w. Then, Ht;!! = 1 and by Lemma 3.1 applied for a = 1/8, 

Pr[|b||oo > n-'^/^] < k^l'^jn^l^-^. 

by /(i) = n - v1 < n ■ n~^/^ = n^/^ 



Now suppose that Halloo 

< n~^/^. Define f : [n] - 
Then, 

\\AsA^)f = {n/s)\\Vs{v)f = = 7E/(^)' 



B. 



and IEtg„[n] /(^) = i^/n) n ■ vf = 1. Therefore, by Corollary 2.3, 



Pr[| 11^ 



Pr 

s 



E fii] 



> e 



< 6. 



The claim now follows. 



□ 



We now recursively apply the above lemma. Fix e,6 > 0. Let A{n, k) : M" — t- M^^") be the collec- 
tion of transformations {As^x ■ S S,x D} as in the above lemma for s{n) = s{n, 
Cn"^/^ log (l/(5)/e^. Note that we can sample from A{n,k) using 0(A;logn + log(l/e) + log(l/5)) 
random bits. Let no = n, and let nj+i = s{ni). Let ko = 16(c + 1) (recall that 5 > \/n^) and 
fcj+i = 2*/co- The parameters nj, ki are chosen so that l/n^^' is always polynomially small. Fix t > 

1/8 

to be chosen later so that ki < n- for i < t. 

Lemma 3.3. For Aq ^(no,A;o),^i A{ni, ki), • • • €« A{nt-i,kt~i) chosen indepen- 

dently, and w G M", \\w\\ = 1, 

Pr[ (1 - eY < ...A,Ao{w)f<{l + eY]>l-t6-Y, ^J/^' 
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Proof. The proof is by induction oni = 1, . . . ,t. For i = 1, the claim is same as Lemma 3.2. Suppose 
the statement is true for i — 1 and let v = Ai^i • • • Aq{w). Then, v G M"' and by Lemma 3.2 applied 
to A{ni,ki), 

Pr[(l-e)|bf < \\A,{v)\\^ < {l + e)\\vf] >l-6 



k,/8-l ' 

n. 



The claim now follows by induction. □ 

What follows is a series of simple calculations to bound the seed-length and error from the 
above lemma. Observe that 

„(./..• < „. = „<./«• , ^ ^ciog(iM)^ = ^^^^j 

Let t = O(loglogn) be such that 2* = log n/8 log log n. Then, nt < log^ n • (Clog(l/(5)/e^)^, 
and for i < t, 

ki<kt = 16(c + 1)2* = 2(c + 1) log n/ log log n < log n = n(^/2)V8 < < ^^^^ (3 2) 

where we assumed that log log n > 2c + 2. Therefore, the error in Lemma 3.3 can be bounded by 



t-l jk,/2 t-l 



t5 + ^ -^-J^ < t5 + n ^ n. ^''^^ (Equation 3.2) 

i=0 i=0 

< M + n|]n~(i/2)'-i6(-+i)-27i6 (Equation 3.1) 

<t5 + t/n^ <2t5 (as (5 > 1/?!^). 

Note that, 

fcilogn./ < 16(c+l)-2^(logn/2^+2Cloglog(l/(5)+41og(l/e)) = 0(logn+lognlog(l/e)/loglogn). 

Therefore, the randomness needed after t = O (log log n) iterations is 
t-l 

^©(fc/logn,/ + log(l/e) +log(l/(5)) = 0(log ?i • log(log n/e)). 
i=0 

Combining the above arguments (and simplifying the resulting expression for seed-length) we get: 

Theorem 3.4. There exists an explicit generator that takes 0{\og{n/ 5) • log( log(n/(5)/e )) random 
hits and outputs a linear transformation A : M" — )• for m = 0((log^'^(n/(5))/e^), so that for 
every w € M", \\w\\ = 1, 

Pr[| Pwf - 1 I > (Clog log n)e] < (Clog log ?i)5. 

We can now obtain our final construction of explicit Johnson-Lindenstrauss families by com- 
posing the above family with that of Theorem 2.4. 
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Proof of Theorem 1.2. Follows by composing the transformations of the above theorem for e' = 
e/Cloglogn, 5' = 5/Cloglogn with those of Theorem 2.4 using 0(log(l/5))-wise independence. 
The additional randomness required is 0(log(l/(5) log m) = 0(log(l/5)(log log(n/(5) +log(l/e)). 

We next bound the time for computing matrix- vector products for the matrices we output. Let 
6 > Note that for i < t, the matrices Ai of Lemma 3.3 are of the form Vs ■ HrnD[x) for a 

/c-wise independent string x G {1,-1}"'. Thus, for any vector Wi E M"', AiWi can be computed 
in time 0(nj log n^) using the discrete Fourier transform. Therefore, for any w = wq & M"", the 
product At-i ■ ■ ■ AiAqWq can be computed in time 

i-l t-i 

^O(nilogni) < 0{n\ogn) + \ogn-^0 [iT^''^\\og(l/6)/e^f^ (Equation 3.1) 

1=0 i=l 

= 0{n\ogn + A/nlognlog^(l/5)/e^). 

It is easy to check that the above bound dominates the time required to perform the final embedding. 

A similar calculation shows that for indices i £ s,j £ [n], the entry G{y)ij of the generated 
matrix can be computed in space O (X^jlognj) = 0(logn + log(l/e) • loglogn) by expanding the 
product of matrices and enumerating over all intermediary indices. The time required to perform 
the calculation is 0{s ■ rit ■ m-i • • • no) = • (log n/e)*^(^°§^°s«). □ 
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